Image processing apparatus, image processing system, image processing method, and image processing computer program

ABSTRACT

An image processing apparatus according to the present invention includes a first identifying unit configured to identify the position of at least a part of a layer boundary based on a tomography image of a target to be captured, a setting unit configured to set a search range for a portion whose position has not been identified by the first identifying unit based on a depth directional position of the layer boundary whose position has been identified by the first identifying unit, and a second identifying unit configured to identify the position of a layer boundary portion whose position has not been identified based on a luminance value in the search range having been set.

TECHNICAL FIELD

The present invention relates to an image processing apparatus, an imageprocessing system, an image processing method, and an image processingcomputer program, which can be used to identify a layer structure basedon a tomography image.

BACKGROUND ART

The eye examination is widely known as an effective method capable ofdiagnosing at early timing the type of a disease that may lead to alifestyle related disease or blindness. An optical coherence tomography(OCT) or a comparable tomography imaging apparatus enables an eye doctoror any other specialist to observe a three-dimensional state of internalretinal layers.

For example, the retina of an ocular fundus has a layer structure thatis composed of a plurality of layers. Information relating to the layerstructure (e.g., thickness of each layer) is usable as an objectiveindex that indicates the stage of a disease.

In order to observe the retinal structure or to obtain the index, atechnique capable of analyzing a tomography image of retinal layers isused to identify the type of a layer boundary.

A conventional method discussed in Japanese Patent Application Laid-OpenNo. 2008-73099 includes performing preprocessing (e.g., gradationconversion) on a tomography image and detecting an edge from theprocessed image in the depth direction. The above-described conventionalmethod further includes identifying the position of a layer boundarybased on the position of a detected edge.

The retinal layer boundary position identification processing performedaccording to the above-described conventional method includesinterpolating an underlying area positioned beneath a blood vessel,which is a representative area where the signal intensity is weakened.The interpolation processing is performed in such a way as to smoothlyconnect an unidentified layer boundary to an already identified retinallayer boundary.

However, the layer boundary interpolation processing discussed inJapanese Patent Application Laid-Open No. 2008-73099 does not take asignal component in the underlying area positioned beneath the bloodvessel, in which a target layer to be interpolated may be present, intoconsideration. Therefore, the reliability of an interpolated layerboundary position is low.

CITATION LIST Patent Literature

-   PTL 1: Japanese Patent Application Laid-Open No. 2008-73099

SUMMARY OF INVENTION

According to an aspect of the present invention, an image processingapparatus includes a first identifying unit configured to identify theposition of at least a part of a layer boundary based on a tomographyimage of a target to be captured, a setting unit configured to set asearch range for a portion whose position has not been identified by thefirst identifying unit based on a depth directional position of thelayer boundary whose position has been identified by the firstidentifying unit, and a second identifying unit configured to identifythe position of a layer boundary portion whose position has not beenidentified based on a luminance value in the search range having beenset.

According to the image processing apparatus having the above-describedconfiguration, if at least a portion of a layer boundary has not beenidentified by the first identifying unit, the second identifying unitidentifies the position of an unidentified layer boundary portion basedon a luminance change in the search range. Thus, the present inventioncan improve the reliability of the image processing apparatus. Further,the image processing apparatus determines a search range based on adepth directional position of the layer boundary. Therefore, the presentinvention can improve the reliability of a layer boundary position to beidentified.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 illustrates an example configuration of an image processingsystem according to the first exemplary embodiment of the presentinvention.

FIG. 2 illustrates an example configuration of a tomography imageacquisition apparatus according to the first exemplary embodiment of thepresent invention.

FIG. 3A schematically illustrates an example tomography image of retinallayers captured in a region including a macula.

FIG. 3B schematically illustrates another example tomography image ofretinal layers captured in a region including a macula and a bloodvessel positioned adjacent to each other.

FIG. 4 is a flowchart illustrating an example flow of processing thatcan be performed by an image processing apparatus according to the firstexemplary embodiment of the present invention.

FIG. 5A illustrates an example of a converted image that can begenerated by an image conversion unit according to the first exemplaryembodiment of the present invention. FIG. 5A schematically illustratesan example image including edges enhanced in the direction from thelower pixel value side to the higher pixel value side.

FIG. 5B illustrates an example of a converted image that can begenerated by an image conversion unit according to the first exemplaryembodiment of the present invention. FIG. 5B schematically illustratesan example image including edges enhanced in the direction from thehigher pixel value side to the lower pixel value side.

FIG. 6 illustrates an example of profiles that can be generated by aluminance information generation unit according to the first exemplaryembodiment of the present invention.

FIG. 7 is a flowchart illustrating an example flow of processing thatcan be performed by a template selection unit and a layer boundaryidentifying unit according to the first exemplary embodiment of thepresent invention.

FIG. 8 schematically illustrates example processing that can beperformed by the template selection unit according to the firstexemplary embodiment of the present invention.

FIG. 9 schematically illustrates example processing that can beperformed by the layer boundary identifying unit according to the firstexemplary embodiment of the present invention.

FIG. 10 schematically illustrates example processing that can beperformed by the layer boundary identifying unit according to the firstexemplary embodiment of the present invention.

FIG. 11 is a flowchart illustrating an example flow of processing thatcan be performed by a layer boundary interpolation unit according to thefirst exemplary embodiment of the present invention.

FIG. 12 schematically illustrates example processing that can beperformed by the layer boundary interpolation unit according to thefirst exemplary embodiment of the present invention.

FIG. 13 is a flowchart illustrating an example flow of processing thatcan be performed by the image processing apparatus according to a secondexemplary embodiment of the present invention.

FIG. 14 is a flowchart illustrating an example flow of processing thatcan be performed by the template selection unit and the layer boundaryidentifying unit according to the second exemplary embodiment of thepresent invention.

FIG. 15 schematically illustrates example processing that can beperformed by the template selection unit according to the secondexemplary embodiment of the present invention.

FIG. 16 schematically illustrates example processing that can beperformed by a determination unit according to the second exemplaryembodiment of the present invention.

FIG. 17 schematically illustrates example processing that can beperformed by the determination unit according to the second exemplaryembodiment of the present invention.

FIG. 18 is a flowchart illustrating an example flow of processing thatcan be performed by the image processing apparatus according to a thirdexemplary embodiment of the present invention.

FIG. 19 schematically illustrates example pattern matching processingthat can be performed by the template selection unit according to thethird exemplary embodiment of the present invention.

FIG. 20 schematically illustrates example pattern matching processingthat can be performed by the template selection unit according to thethird exemplary embodiment of the present invention.

FIG. 21 is a flowchart illustrating an example flow of processing thatcan be performed by the layer boundary interpolation unit according to afourth exemplary embodiment of the present invention.

FIG. 22 illustrates an example of search range setting that can beperformed by the layer boundary interpolation unit according to a fourthexemplary embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

An image processing system (optical coherence tomography imaging system)100 according to a first exemplary embodiment includes an imageprocessing apparatus 101 that can identify the position and the type ofa layer structure based on a tomography image of a retina received froma tomography image acquisition apparatus 102.

In this case, the image processing apparatus 101 identify layerboundaries by applying template information to each portion having aunique structure in the tomography image.

Further, the image processing apparatus 101 interpolates an unidentifiedlayer boundary portion, if it was not identified by the applied templateinformation, with reference to the position of the layer boundaryalready identified based on the template information.

In this case, the image processing apparatus 101 sets a range that canbe presumed to include the unidentified layer boundary portion based onthe position of the already identified layer boundary. Then, the imageprocessing apparatus 101 identifies a target layer boundary based on aluminance change value in the set range.

An example configuration of the image processing system 100 and exampleprocessing that can be performed by the image processing system 100 aredescribed below.

An example configuration of the image processing system 100 is describedbelow with reference to FIG. 1. The image processing apparatus 101 is,for example, a computer. The image processing apparatus 101 includes animage acquisition unit 103, an image conversion unit 104, a luminanceinformation generation unit 105, a detection unit 106, a structuredetermination unit 1071, a display control unit 110, and a storage unit111.

Each of the above-described functional blocks can be constituted by anelectric circuit. Alternatively, the image processing apparatus 101 mayhave a hardware configuration and a software configuration that arecooperatively operable as each of the above-described functional blocks.

As an example software configuration (although not illustrated in thedrawing), the image processing apparatus 101 can include a centralprocessing unit (CPU), a read only memory (ROM), and a random accessmemory (RAM). For example, to realize processing described in FIG. 4,FIG. 7, and FIG. 11, the CPU can execute a program or programs loadedinto the RAM from the ROM. In this manner, the hardware and softwareconfiguration of the computer can cooperatively realize the presentinvention.

The tomography image acquisition apparatus 102 is, for example, anoptical coherence tomography imaging apparatus (optical coherencetomography imaging unit) whose operation is based on the principle ofOptical Coherence Tomography. The tomography image acquisition apparatus102 is described below in more detail.

The image acquisition unit 103 of the image processing apparatus 101acquires a tomography image from the tomography image acquisitionapparatus 102. The tomography image obtainable from the tomography imageacquisition apparatus 102 is a tomography image of a three-dimensionalregion of a retina that can be obtained by scanning a predeterminedtwo-dimensional area on a retinal surface.

The tomography image represents an internal layer structure of theretina. The tomography image can be acquired as a plurality oftwo-dimensional tomography images (B-scan images) or can be acquired asone-dimensional images (A-scan images) obtained at a plurality ofpositions on the retinal surface irradiated with signal light.Alternatively, the tomography image may be acquired as three-dimensionalvolume data that can be generated based on A-scan images.

The image conversion unit 104 can use a Sobel filter and a median filterto obtain images converted from an input image. More specifically, whenthe Sobel filter is applied to an acquired tomography image, a Sobelimage having enhanced edges can be obtained. When the median filter isapplied to an acquired tomography image, a smoothed image can begenerated as a median image.

The luminance information generation unit 105 can generate, from theSobel image and the median image, information indicating a relationshipbetween luminance value and depth directional position at apredetermined position in the horizontal direction. More specifically,the information to be generated by the luminance information generationunit 105 is a profile in the depth direction at a predetermined positionin the horizontal direction.

The profile is information indicating a relationship between depthdirectional position and luminance value. The width of the profile inthe horizontal direction is one pixel. Original data that can be used togenerate the profile is a row of pixels disposed in the depth direction.As described above, the Sobel image is an image having enhanced edges. Aluminance value of the Sobel image represents a luminance change in theoriginal image.

The predetermined position where a profile is generated is not limitedto a one-dimensional image region. For example, the above-describedpredetermined position can be a two-dimensional image area that has awidth corresponding to several pixels in the horizontal direction andincludes pixels disposed in the depth direction.

Alternatively, the above-described predetermined position can be athree-dimensional image area. A layer boundary that is present in thedepth direction at a position where the profile is created in thehorizontal direction can be identified.

In this case, the horizontal direction is a direction perpendicular tothe direction of irradiated signal light (A-scan direction). It isdesired that the above-described horizontal direction coincides with thehorizontal direction of an image. In almost all of a normal eye, exceptfor the macula and the optic disc, the above-described horizontaldirection is a direction along which the layer extends and a directionparallel to the layer.

Further, the above-described depth direction is the direction ofirradiated signal light (A-scan direction). It is desired that the depthdirection coincides with the vertical direction of an image. In anordinary display, the above-described A-scan direction is set to becoincident with the vertical direction of a screen.

A retinal layer has a multilayered structure that includes a pluralityof layers stacked on top of another in the depth direction. Therefore,in almost all of a normal eye except for the macula and the optic disc,a direction along which two or more layers are sequentially positionedis the depth direction.

The luminance information generation unit 105 generates profiles atpredetermined intervals (e.g., at intervals of five pixels) in thehorizontal direction. Identification of a layer boundary according tothe present invention is not performed at a position other than theposition where the profile is created. Any conventional interpolationmethod can be used to obtain an unidentified layer boundary referring tothe position of an already identified layer boundary that has beenidentified based on the profile.

To detect a layer boundary, the detection unit 106 acquires, based onthe profile of a Sobel image at each position, a feature point thatindicates the position of each layer boundary of layers that aresequentially positioned in the depth direction.

In the present exemplary embodiment, the detection unit 106 acquires, asthe feature point, an edge which is greater than a predeterminedthreshold value, among edges detectable along the profile in the depthdirection.

In the present exemplary embodiment, each edge represents the gradientof a luminance value in an image. The above-described predeterminedthreshold value is an experimentally determined value and corresponds toa first threshold value according to the present invention.

In the present exemplary embodiment, the layer boundary indicates aninterface between two neighboring layers each having a predeterminedthickness in a tomography image.

If the resolution of an image is sufficiently high, a layer having athickness less than one pixel may be detected as a boundary and can beregarded as a boundary.

The first threshold value can be modified and determined for eachprofile based on a statistic value calculated based on a profile. Thedetection unit 106 performs the above-described detection processing inthe A-scan direction (i.e., in the depth direction) at a plurality ofpositions of a tomography image in the horizontal direction.

The structure determination unit 1071 can determine a structureaccording to the number of layer boundaries detected by the detectionunit 106. The structure determined by the structure determination unit1071 represents a layer structure that can be defined by the number oflayers or layer boundaries and the position and the type thereof. Thestructure further includes a pseudo-image or a lesion caused by a bloodvessel or a leucoma, in addition to the optic disc, the macula, andother regions.

In the present exemplary embodiment, the structure determination unit1071 corresponds to a determination unit configured to determine theposition and the type of a layer boundary. The structure determinationunit 1071 includes a template selection unit 107, a layer boundaryidentifying unit 108, and a layer boundary interpolation unit 109.

The template selection unit 107 allocates template information to eachprofile according to the number of feature points each representing theposition of a layer boundary acquired from the profile. The templateinformation is information representing the type and the position ofeach layer boundary as well as a relationship between layer boundaries,i.e., layer areas, in the magnitude of luminance value. The templateinformation represents the type of a plurality of layer boundaries thatare sequentially disposed in the depth direction at each of a pluralityof positions of a target to be captured in the horizontal direction.

The storage unit 111 stores a plurality of types of templates that aredifferentiated according to the structure of each retinal layer orconsidering the presence of a blood vessel. The structure determinationunit 1071 identifies the position and the type of a target layerboundary based on the profile with reference to the above-describedtemplate information.

The template information is associated with information indicating thenumber of feature points, when it is stored. Thus, the structuredetermination unit 1071 can determine an appropriate template based onfeature points with reference to the template information.

The template can be created by extracting information indicating theposition and the type of a layer boundary and information indicating therelationship between layer boundaries in the magnitude of luminancevalue from a reference profile that can be created by obtaining anaverage value or a central value from a plurality of tomography imagescapturing the same portion of a layer boundary whose position and typehave been identified beforehand. The template may include informationindicating a luminance value of each layer.

For example, the structure of a retinal layer at the macula or at theoptic disc is different from the structure of the retinal layer at otherportions. Therefore, the templates prepared in the storage unit 111include specific templates that correspond to the macula or the opticdisc.

Further, if a blood vessel is present at a predetermined position of aretinal layer, signal light is absorbed by the blood vessel. Theintensity of the signal light is weakened when it reaches an areabeneath the blood vessel. Therefore, the profile may be deformeddepending on the presence of a blood vessel.

Therefore, for example, two or more different templates are prepared forthe same macula considering the above-described situation (i.e.,according to the presence of a blood vessel). It is desired to preparevarious types of templates so that at least one template can be appliedto any predicable situation (e.g., presence of a lesion).

Experimentally creating template information based on tomography imagesof numerous eyes to be examined is feasible. Further, to reduce theprocessing time, it may be useful to limit the number of selectableprofiles. The above-described templates are stored in the storage unit111. Examples of the template are described below in more detail.

The layer boundary identifying unit 108 is functionally operable as anidentifying unit or a first identifying unit configured to identify alayer structure in a tomography image by applying a template to eachprofile.

In the present exemplary embodiment, identification of each layerstructure includes identifying the position of the layer structure orthe type thereof. The information indicating each identified layerboundary is associated with the profile or the tomography image, and canbe stored in the storage unit 111 or in a data server 113. Example ofthe above-described identifying processing is described below in moredetail.

The layer boundary interpolation unit 109 is functionally operable as asetting unit configured to set a search range for a layer boundaryportion that has not been identified based on the application of atemplate. Further, the layer boundary interpolation unit 109 isfunctionally operable as an identifying unit or a second identifyingunit configured to interpolate a layer boundary by identifying theposition of an unidentified layer boundary portion.

In the present exemplary embodiment, the layer boundary interpolationunit 109 correlates interpolation completion information with theinformation indicating the interpolated layer boundary position.Further, the layer boundary interpolation unit 109 stores theinformation indicating the interpolated layer boundary position inassociation with the profile or the tomography image in the storage unit111 or in the data server 113.

An example of the above-described interpolation processing is describedbelow in detail with reference to a flowchart illustrated in FIG. 11.

The display control unit 110 can control a display unit 112 to display atomography image together with information indicating the type andposition of each layer boundary. In the above-described control, thedisplay control unit 110 causes the display unit 112 to use a uniquecolor for each layer boundary included in the tomography image accordingto the type of each layer.

Further, it is useful that the display control unit 110 controls thedisplay unit 112 to differentiate a display pattern for the position ofa layer boundary identified based on template information from a displaypattern for the position of a layer boundary identified by interpolationprocessing.

The storage unit 111 stores information to be required by respectiveblocks of the image processing apparatus 101 and information output fromrespective blocks. For example, the detection unit 106 can use the firstthreshold value or its calculation method stored in the storage unit111. The template selection unit 107 can use template information storedin the storage unit 111. The layer boundary interpolation unit 109 canuse a search range setting value and a second threshold value stored inthe storage unit 111.

The display unit 112 is, for example, a liquid crystal display device,which can display a tomography image output by the image processingapparatus 101 together with the position and the type of each layerboundary.

The data server 113 is functionally operable as a storage unitconfigured to store tomography images acquired by the tomography imageacquisition apparatus 102 together with bibliographic information.

An example configuration of the tomography image acquisition apparatus102 is described below in more detail with reference to FIG. 2. Thetomography image acquisition apparatus 102 is a so-called Fourier-Domaintype optical coherence tomography imaging apparatus. The tomographyimage acquisition apparatus 102 can capture an image of an eye to beexamined (not illustrated) in response to an instruction of an operator(not illustrated). The tomography image acquisition apparatus 102 cantransmit a captured image to the image processing apparatus 101 and tothe data server 113.

A light source 114 emits light. A beam splitter 115 is capable ofdividing the light emitted from the light source 114 into measurementlight 124 and reference light 125. The measurement light 124 reaches aneye 118 (i.e., a target to be observed) and returns as feedback light126. The feedback light 126 (i.e., the light returning from the eye 118)includes reflection light and scattered light of the measurement light.

Further, the beam splitter 115 is functionally operable as a coherentlight generation unit configured to multiplex the feedback light 126 andthe reference light 125 to generate coherent light 127. A diffractiongrating 120 disperses the coherent light 127. The dispersed coherentlight passes through a lens 121 and forms an image on a one-dimensionalsensor 122.

The one-dimensional sensor 122 includes a plurality of pixel circuitseach outputting an electric signal that represents the quantity ofreceived light. An image formation unit 123 performs Fouriertransformation with reference to an internal position of theone-dimensional sensor 122 (i.e., the number of waves of the coherentlight), and obtains a tomography image of the eye 118.

The light source 114 is described below in more detail. The light source114 is a super luminescent diode (SLD), which is known as arepresentative low coherent light source. The light emitted from thelight source 114 has a wavelength of 830 nm. The light emitted from thelight source 114 has a bandwidth of 50 nm.

The bandwidth of the light emitted from the light source 114 is animportant parameter because the bandwidth significantly influences theresolution of an obtained tomography image in the optical axisdirection. Further, the type of the light source 114 is not limited tothe above-described SLD. Any other type of light source can be used asthe light source 114, if it can emit low coherent light. For example,amplified spontaneous emission (ASE) can be used as the light source114.

Further, near infrared light is usable as the light to be emitted fromthe light source 114 because the near infrared light has a wavelengtheffective to measure an eye. Further, the wavelength significantlyinfluences the resolution of an obtained tomography image in thehorizontal direction. Therefore, it is desired to set the wavelength asshort as possible. In the present exemplary embodiment, the selectedwavelength is 830 nm. Another wavelength may be selected depending onthe portion of an observation target to be measured.

An optical path of the reference light 125 is described below. Thereference light 125 (i.e., one light component separated (dispersed) bythe beam splitter 115) is reflected by a minor 119 (i.e., a referenceobject) and returns to the beam splitter 115. When the length of theoptical path of the reference light 125 is equal to the length of anoptical path of the measurement light 124, the reference light 125 caninterfere with the measurement light 124.

Next, the optical path of the measurement light 124 is described. Themeasurement light 124 (i.e., the other light component separated(dispersed) by the beam splitter 115) reaches a mirror of an XY scanner116, which can change the direction of the measurement light 124 towardthe eye 118. The XY scanner 116 is functionally operable as a scanningoptical system that performs two-dimensional raster scanning on a retinaof the eye 118 in a direction perpendicular to the optical axis bysuccessively changing the direction of the measurement light 124.

Although not illustrated in the drawing, the XY scanner 116 is composedof two minors (i.e., an X-scanning minor and a Y-scanning minor) thatare positioned adjacent to each other. Further, the measurement light124 and the XY scanner 116 are adjusted beforehand so as to satisfy thepositional relationship that the center of the measurement light 124coincides with the rotational center of the mirror of the XY scanner116.

The measurement light 124, after passing through a lens 117, isconverged onto the retina. When the measurement light 124 reaches theeye 118 through the above-described optical system, the measurementlight 124 becomes the feedback light 126 when reflected and scattered bythe retina of the eye 118.

In the present exemplary embodiment, generating a one-dimensional imagewith the measurement light 124 reaching a point on the retina isreferred to as “A-scan” processing and the generated one-dimensionalimage is referred to as “A-scan image.” Further, generating atwo-dimensional image by performing the A-scan processing atpredetermined intervals along a predetermined line on the retinalsurface is referred to as “B-scan” processing and the generatedtwo-dimensional image is referred to as “B-scan image.”

A plurality of A-scan images can be obtained at a plurality of positionswhen the B-scan is performed at predetermined intervals by successivelychanging the incidence position of the measurement light. Atwo-dimensional B-scan image can be obtained by performing interpolationprocessing on the obtained plurality of A-scan images.

Further, an ordinary OCT apparatus includes a scanning-type laserophthalmoscope (not illustrated) capable of monitoring animage-capturing position or an optical system capable of capturing atwo-dimensional image of an ocular fundus.

Next, an example spectroscopic system is described. As described above,the coherent light 127 is separated (dispersed) by the diffractiongrating 120. The above-described dispersion is performed under the samewavelength conditions as the central wavelength and the bandwidth of thelight source. Further, the one-dimensional sensor to be used to measurethe coherent light is generally a charge-coupled device (CCD) typesensor or a complementary metal oxide semiconductor (CMOS) type sensor.

The image processing apparatus 101 analyzes a tomography image capturedby the above-described tomography image acquisition apparatus 102 (i.e.,the optical coherence tomography imaging apparatus). In the presentexemplary embodiment, the tomography image acquisition apparatus 102 maynot be the optical coherence tomography imaging apparatus itself and maybe functionally operable as an apparatus that can acquire a tomographyimage from the data server 113 if the data server 113 stores tomographyimages captured by the optical coherence tomography imaging apparatus.

Next, an example of the tomography image of retinal layers that can beacquired by the above-described tomography image acquisition apparatus102 is described below with reference to FIGS. 3A and 3B. The tomographyimage illustrated in FIG. 3 has a layer structure that can be identifiedby the image processing apparatus 101.

FIGS. 3A and 3B illustrate examples of the structure of retinal layers.The layer structure identification processing according to the presentexemplary embodiment can be applied to the retinal layers illustrated inFIGS. 3A and 3B.

FIG. 3A schematically illustrates an example tomography image of retinallayers captured in a region including the macula, in which each solidline represents a layer boundary. Each of a plurality of two-dimensionaltomography images (B-scan images, hereinafter referred to as “tomographyimages”) T1 to Tn, each including a macula image, can be obtained byperforming the A-Scan processing along a line on the retinal surface soas to obtain a two-dimensional image.

The above-described A-scan processing is discretely (i.e.,discontinuously) performed at predetermined intervals along a line. Anappropriate interpolation method can be employed to obtain aninterpolated two-dimensional image that corresponds to an intermediatepoint between two neighboring discrete points where the A-scanprocessing has been performed.

The tomography image Tn includes an interface L1 of an inner limitingmembrane (ILM), a nerve fiber layer (NFL) L2′, and a boundary L2 betweenthe nerve fiber layer L2′ and an underlying layer thereof. Further, thetomography image Tn includes a boundary L3 between an inner plexiformlayer and an underlying layer thereof and a boundary L4 between an outerplexiform layer and an underlying layer thereof. Further, the tomographyimage Tn includes a boundary L5 of an interface between inner and outersegments of the photoreceptors (IS/OS) and a lower boundary L6 of aretinal pigment epithelium (RPE). As described above, the retina has alayer structure that is composed of a plurality of layers that aresequentially positioned in the depth direction.

Discriminating the boundary between the IS/OS and the RPE may bedifficult if the performance of the OCT imaging apparatus isinsufficient, although the above-described detection accuracy issufficient enough to realize the present invention.

Further, the inner limiting membrane (ILM) is a thin layer, although ithas a predetermined thickness. Similarly, the interface between innerand outer segments of the photoreceptors (IS/OS) is a thin layer.Therefore, these layers are recognized as lines when an image isdisplayed.

Therefore, in a case where a low-resolution image is processed,discriminating the interface of the inner limiting membrane from theinner limiting membrane itself may be difficult. In this case,identifying the interface of the inner limiting membrane is notdifferent from identifying the inner limiting membrane itself. Further,the layer boundary is substantially identical to the interface.

In general, the retina has a layer structure that is composed of aplurality of retinal layers as described above. However, the retinallayer structure may be different or modified if the position to beexamined is changed or if a lesion is present.

FIG. 3B schematically illustrates another tomography image of retinallayers in a region including the macula. The tomography imageillustrated in FIG. 3B includes a blood vessel V1. In this case, redblood cells contained in the blood vessel attenuate the signal light.Therefore, in a horizontal area S2 in which the blood vessel V1 ispresent, a pseudo-image may be generated because it is difficult tocapture an image of an underlying area positioned beneath the bloodvessel V1.

Accordingly, an apparent structure that can be identified based on thetomography image in the area S2 may be different from that obtainablefrom an image of an area S1 in which no blood vessel is present.Similarly, if a leucoma is present, a pseudo-image may be generatedbecause it is difficult to capture an image of an underlying areapositioned beneath the leucoma.

The region where the above-described lesion or similar abnormalityappears, i.e., the region where the structure changes greatly, is avertical layer zone extending from the inner limiting membrane L1 toboundary L6 of the retinal pigment epithelium. Therefore, afteridentifying the above-described layers, their inner layers areidentified.

An example flow of processing that can be performed by the imageprocessing apparatus 101 having the above-described configuration isdescribed below with reference to a flowchart illustrated in FIG. 4. Theprocessing according to the present exemplary embodiment is identifyingthe interface of the inner limiting membrane L1 and the interfacebetween inner and outer segments of the photoreceptors (IS/OS).

The inner limiting membrane L1 or the interface between inner and outersegments of the photoreceptors (IS/OS) includes a boundary betweenanother layer and the layer itself.

In the present exemplary embodiment, processing is performed to identifythe position of an interface positioned on the upper side when seen inthe depth direction because an edge from the lower side to the upperside is seen.

If the resolution of an image is insufficient, the width of theabove-described layer may be narrower than one pixel of the image. Inthis case, it may be difficult to determine whether the positionidentified as a layer boundary represents an upper-side interface, alower side interface, or the layer itself. In such a case, identifyingan interface of a layer is not different from identifying the layeritself.

(Step S401) In step S401, the image acquisition unit 103 acquires an OCTimage from the tomography image acquisition apparatus 102.

(Step S402) In step S402, the image conversion unit 104 performs imageconversion processing on the OCT image acquired by the image acquisitionunit 103.

In the present exemplary embodiment, the image conversion unit 104applies the median filter and two types of Sobel filters to the acquiredtomography image. As described above, an image obtained through theabove-described conversion processing is referred to as a median imagewhen the median filter is used.

One of the above-described two types of Sobel filters is a Sobel filterthat can enhance an edge from a low-luminance pixel to a high-luminancepixel in the depth direction of the A-scan line. The other of theabove-described two types of Sobel filters is a Sobel filter that canenhance an edge from a high-luminance pixel to a low-luminance pixel.Hereinafter, when converted images are obtained through theabove-described conversion processing using the Sobel filters, theobtained images are sequentially referred to as Sobel image A and Sobelimage B.

FIG. 5A illustrates an example of the Sobel image A, which can beobtained by applying one of the above-described two types of Sobelfilters to an OCT image. The Sobel image A illustrated in FIG. 5Aincludes enhanced edges of the inner limiting membrane and the interfacebetween inner and outer segments of the photoreceptors.

FIG. 5B illustrates an example of the Sobel image B, which can beobtained by applying the other of the above-described two types of Sobelfilters to the same OCT image. The Sobel image B illustrated in FIG. 5Bincludes enhanced edges of the nerve fiber layer boundary, the innerplexiform layer, the outer plexiform layer, and the retinal pigmentepithelium.

The median image, the Sobel image A, and the Sobel image B obtainedthrough the image conversion processing in the step S402 are stored inthe data server 113.

The image conversion method is not limited to the above-describedmethod. For example, the above-described median filter can be replacedby a smoothing filter (e.g., an average value filter). Further, theimage conversion processing according to the present exemplaryembodiment can be realized by using a gradation conversion filter(capable of performing gamma correction) or a morphology filter insteadof using the smoothing filter or an edge enhancement filter.

Alternatively, a luminance value of the original image can be directlyused as an input for the next step when the above-described imageconversion processing is skipped.

(Step S403) In step S403, the luminance information generation unit 105generates luminance information based on the converted image obtained instep S402.

In the present exemplary embodiment, the luminance informationgeneration unit 105 checks the luminance value in the depth direction,on a pixel by pixel basis, in an image area positioned at apredetermined position. Further, the luminance information generationunit 105 generates a profile that represents the obtained luminanceinformation.

The above-described predetermined position is a one-dimensional imageregion extending along the A-scan line, i.e., in the depth direction.

The A-scan line indicates a row of pixels disposed in the depthdirection of the image. The depth direction of a profile image coincideswith the direction of the axial scanning (A-scan processing) performedon an OCT image. However, the row of pixels may not be a row of pixelsthat corresponds to the position where the A-scan processing has beenperformed.

In the present exemplary embodiment, the A-scan line scanning processingis performed at intervals of five pixels on a tomography image having awidth of 256 pixels and a height of 250 pixels. Therefore, fifty A-scanlines are set for a piece of B-scan image. Each of the fifty A-scanlines is designated as a target to be identified with respect to theretinal layer boundary.

The luminance information generation unit 105 generates a profile alongeach of the above-described A-scan lines. The above-described processingis performed on each converted image obtained in the previous step andthe obtained data is stored in the data server 113 (or may be stored inthe storage unit 111).

However, the luminance information generation method is not limited tothe above-described method that is characterized in that generation ofthe luminance information is performed on a pixel by pixel basis. Forexample, the luminance information can be generated based on a blockarea composed of a plurality of pixels. Further, the scanning processingalong the A-scan line may be performed at different intervals.

(Step S404) In step S404, the detection unit 106 acquires a feature areabased on the luminance information created in step S403.

In the present exemplary embodiment, the detection unit 106 checks theprofile generated based on the Sobel image A. Then, the detection unit106 acquires an area whose luminance is equal to or greater than apredetermined threshold value as the feature area. The Sobel image is animage including an enhanced edge. The area to be detected by thedetection unit 106 is an area in which the luminance change in the depthdirection is equal to or greater than a predetermined threshold value.Hereinafter, the area acquired by the detection unit 106 is referred toas a peak area or a peak.

Example processing to be performed to detect a peak from a tomographyimage is described below with reference to FIG. 6. In the OCT imageillustrated in FIG. 6, a vertical line A6 represents one of the A-scanlines (i.e., a row of pixels). The OCT image illustrated in FIG. 6further includes a profile PSA6 of the Sobel image A taken along theA-scan line A6 and a profile PSB6 of the Sobel image B taken along theA-scan line A6.

As described in step S302, the profile of the Sobel image A and theprofile of the Sobel image are characterized in that a specific retinallayer boundary is enhanced and the enhanced boundary appears as a peakin the graph representing a change in the luminance.

In the present exemplary embodiment, a threshold Th is set and an areawhose luminance value is equal to or greater than the threshold Th isregarded as a peak.

The above-described threshold value corresponds to the predeterminedthreshold value or the first threshold value according to the presentinvention. As described above, the peak detection processing isperformed along each A-scan line. Peak information (e.g., position andmagnitude) obtained through the above-described processing is stored inthe data server 113.

In the present exemplary embodiment, the position of the peak representsthe position of a local maximum point in a detected peak area and themagnitude of the peak represents the magnitude of the local maximumpoint. The local maximum point in the peak area is regarded as a featurepoint.

In the present exemplary embodiment, the detection unit 106 acquires theabove-described peak position as the position of a layer boundary,although the type of a layer boundary whose position has been identifiedis not yet identified at this moment.

The feature area detection method is not limited to the above-describedmethod. For example, any other point (e.g., a profile maximum point or aprofile minimum point) in the feature area can be used to detect thefeature point.

In steps S405 and S406, the structure determination unit 1071 performsprocessing to determine a structure based on the layer boundary or theedge detected by the detection unit 106. In the present exemplaryembodiment, the structure determination unit 1071 determines the type ofa layer boundary that is present in each of the A-scan lines.

(Step S405) In step S405, the template selection unit 107 roughlyestimates the structure of a retinal layer to be identified based on thefeature points detected in step S404. Then, the template selection unit107 selects an appropriate template that is similar to the detectedlayer structure.

Through the above-described processing, a layer boundary type thatcorresponds to any one of layer boundaries at the peak position acquiredby the detection unit 106 can be determined. Then, the layer boundaryidentifying unit 108 identifies a correspondence relationship inrelation to the type of the detected layer boundary based on theposition acquired in step S404 and the template information.

Detailed processing to be performed in step S405 is described below withreference to the flowchart illustrated in FIG. 7. The flowchartillustrated in FIG. 7 describes processing to be performed for eachA-scan line. If the processing according to the flowchart illustrated inFIG. 7 is thoroughly completed for all of the A-scan lines, theprocessing proceeds to step S406.

(Step S406) In step S406, the layer boundary interpolation unit 109interpolates the retinal layer boundary along each A-scan line fromwhich no retinal layer boundary was identified. Detailed processing tobe performed in step S406 is described below with reference to theflowchart illustrated in FIG. 11.

In the flowchart illustrated in FIG. 11, the layer boundaryinterpolation unit 109 performs branched processing for each of theA-scan lines. If the processing according to the flowchart illustratedin FIG. 11 is thoroughly completed for all of the A-scan lines, theprocessing proceeds to step S407.

(Step S407) In step S407, the display control unit 110 causes thedisplay unit 112 to display interpolated lines that can be obtained byconnecting boundary points of the inner limiting membrane or theinterface between inner and outer segments of the photoreceptors, whichhave been identified at predetermined intervals along each A-scan line.

As described above, when the type of each layer boundary to beidentified is determined with reference to the features than can beextracted from an image, it becomes feasible to eliminate errors inidentifying the type of each layer boundary.

Further, when a template is selected and applied in the above-describedprocessing to identify each layer boundary, the position of the layerboundary can be identified with reference to the features extracted froman image. Therefore, the position of each detected layer boundary can beidentified accurately or reliably.

Next, an example flow of the processing to be performed in step S405,i.e., example processing that can be performed by the template selectionunit 107 and the layer boundary identifying unit 108, is described belowwith reference to FIG. 7.

(Step S701) In step S701, the template selection unit 107 selects a rowof pixels (A-scan line) as a target to be analyzed. More specifically,the template selection unit 107 successively selects a row of pixelsdisposed in the depth direction at intervals of five pixels in thehorizontal direction. The interval in the above-described processing isnot limited to five pixels and can be arbitrarily set.

(Step S702) In step S702, the template selection unit 107 counts thetotal number of the feature points along each A-scan line and determinesthe type of each detected layer boundary (i.e., a target to beidentified). If the total number of the feature points is two, thetemplate selection unit 107 determines that each of the interface of theinner limiting membrane and the interface between inner and outersegments of the photoreceptors as a layer boundary to be identified.Then, the processing proceeds to step S703. If the total number of thefeature points is not two, the processing proceeds to step S705.

Detailed processing is described below with reference to FIG. 8. FIG. 8illustrates an example tomography image of retinal layers, in whichvertical lines A81 and A82 represent two A-scan lines. Further, PSA81and PSA81 represent profiles obtained from the Sobel image A.

To estimate a retinal layer, the template selection unit 107 counts thenumber of peaks appearing on each profile. The tomography imageillustrated in FIG. 8 includes the blood vessel V1 that generates apseudo-image. Due to the presence of the blood vessel V1, the interfacebetween inner and outer segments of the photoreceptors cannot beidentified along or in the vicinity of the A-scan line A81 in thetomography image illustrated in FIG. 8.

Therefore, the template selection unit 107 counts the number of peaksappearing on the profile of the Sobel image A along each A-scan line. Ifthe total number of the counted peaks is two, the template selectionunit 107 determines that the inner limiting membrane and the interfacebetween inner and outer segments of the photoreceptors may be present inthe image when taken along or in the vicinity of the A-scan line.

In this case, the template selection unit 107 determines that the typeof the target (i.e., the layer boundary) to be identified is both theinner limiting membrane and the interface between inner and outersegments of the photoreceptors. Subsequently, the processing proceeds tostep S702.

If only one peak is present, the template selection unit 107 determinesthat only the inner limiting membrane may be present in the image whentaken along or in the vicinity of the A-scan line. In this case, thetemplate selection unit 107 determines that the type of the target(i.e., the layer boundary) to be identified is the inner limitingmembrane. Subsequently, the processing proceeds to step S303.

If the total number of the counted peaks is not the above-describednumber (two or one), the template selection unit 107 determines that anoise may be present. In this case, the template selection unit 107determines that the type of the target (i.e., the layer boundary) to beidentified is nothing. The template selection unit 107 does not identifyany retinal layer boundary along the A-scan line.

As described above, even when a layer boundary is detected based on anedge or edges by the detection unit 106, if an appropriate template isnot present, the template selection unit 107 determines that ananatomical layer boundary type cannot be identified and does not performlayer boundary identification processing because the detected layerboundary is regarded as a noise.

(Step S703) In step S703, the template selection unit 107 selects atemplate from the storage unit 111 according to the number of layerboundaries detected based on feature points (i.e., edges).

(Step S704) The layer boundary identifying unit 108 performs layerboundary identification processing along the A-scan line that waspresumed in step S702 as positioning in an area where both the interfaceof the inner limiting membrane and the interface between inner and outersegments of the photoreceptors are present.

In the present exemplary embodiment, the layer boundary identifying unit108 successively identifies, from a shallow side, the positions of twofeature points as the interface of the inner limiting membrane and theinterface between inner and outer segments of the photoreceptors withreference to the template information.

The layer boundary identifying unit 108 can also perform the followingprocessing. Example processing that can be performed by the layerboundary identifying unit 108 is described below in detail withreference to FIG. 9. FIG. 9 illustrates an example tomography image ofretinal layers, in which a vertical line A9 represents one of the A-scanlines. The tomography image illustrated in FIG. 9 further includes aprofile PSA9 obtained from the Sobel image A and a profile PMD9 obtainedfrom the median image.

Further, a plurality of profiles used in the present step includes areference profile PRE9 that can be derived from the profile obtainedfrom the median image. The reference profile PRE9 is additionallyprepared to identify the boundaries of the target two layers (i.e., theinterface of the inner limiting membrane and the interface between innerand outer segments of the photoreceptors). The reference profile PRE9 isa typical example of the profile taken along the A-scan line extendingin an area where both of the target two layers (i.e., the interface ofthe inner limiting membrane and the interface between inner and outersegments of the photoreceptors) are present.

In the present exemplary embodiment, the typical example indicates atendency of the luminance between respective layers along the A-scanline extending in an area where both of the target two layers (i.e., theinterface of the inner limiting membrane and the interface between innerand outer segments of the photoreceptors) are present, in which no noiseis present.

For example, when the luminance value is observed in an intermediateregion between the interface of the inner limiting membrane and theinterface between inner and outer segments of the photoreceptors, thereis a tendency that the luminance value becomes higher in a regionadjacent to the inner limiting membrane because of the presence of thenerve fiber layer and the inner plexiform layer. The reference profilePRE9 to be prepared in the present exemplary embodiment possesses theabove-described tendency in the luminance features.

The reference profile PRE9 is not limited to the above-describedexample. Any other profile can be employed if the luminance tendency ofeach retinal layer and a positional relationship between retinal layerscan be identified based on the employed profile.

The reference profile PRE9 does not reflect the thickness of eachretinal layer, although the thickness is generally variable depending onthe position of the A-scan line. Therefore, the reference profile PRE9is employable for any A-scan line extending in an area where both of thetarget two layers (i.e., the interface of the inner limiting membraneand the interface between inner and outer segments of thephotoreceptors) are present.

The layer boundary identifying unit 108 uses the reference profile PRE9to determine whether the detected two peaks coincide with theabove-described two layers (i.e., the interface of the inner limitingmembrane and the interface between inner and outer segments of thephotoreceptors)

First, the layer boundary identifying unit 108 calculates an averageluminance value between respective peaks appearing on the profile PSA9.The layer boundary identifying unit 108 uses, as a luminance value, avalue of the profile PMD9 corresponding to the position of the profilePSA9.

The layer boundary identifying unit 108 refers to two peaks of theprofile PSA9 as a first peak and a second peak, respectively, which arepositioned in this order from the shallow side. The layer boundaryidentifying unit 108 calculates the average luminance value in a rangeextending from the first peak to the second peak (hereinafter, referredto as “peak-to-peak A1”). The range “peak-to-peak A1” includes twoequally divided ranges. One of the above-described two divided ranges,which is adjacent to the first peak, is referred to as “peak-to-peakA11.” The other of the above-described two divided ranges, which isadjacent to the second peak, is referred to as “peak-to-peak A12.”

Further, the layer boundary identifying unit 108 calculates an averageluminance value of the background (i.e., the remaining area other thanthe retinal layers) of the image. In the present exemplary embodiment,the layer boundary identifying unit 108 performs binarization processingusing a threshold value that can be experimentally determined for themedian image.

The above-described average luminance value calculation by the layerboundary identifying unit 108 is exclusively performed for a target areawhose luminance value is less than the threshold value. The backgroundaverage luminance value calculation method is not limited to theabove-described method.

The threshold value to be used in the binarization processing can bedetermined according to the discriminant analysis method or thePercentile method (P-tile method). Further, it may be useful that thelayer boundary identifying unit 108 calculates an average luminancevalue using luminance values at an upper edge and a lower edge of animage that does not include any retinal layers.

Next, the layer boundary identifying unit 108 compares the calculatedaverage luminance values to determine a relationship between them in themagnitude while taking the reference profile PRE9 into consideration.Two conditions to be satisfied with respect to the average luminancevalues, which can be derived from the reference profile PRE9, arepeak-to-peak A1>background average luminance value and peak-to-peakA11>peak-to-peak A12.

The layer boundary identifying unit 108 confirms whether the calculatedaverage luminance values coincide with the above-described two layers(i.e., the interface of the inner limiting membrane and the interfacebetween inner and outer segments of the photoreceptors) by checkingwhether the calculated average luminance values satisfy theabove-described conditions.

If the above-described conditions are all satisfied, the layer boundaryidentifying unit 108 identifies the first peak as the interface of theinner limiting membrane and identifies the second peak as the interfacebetween inner and outer segments of the photoreceptors. The identifiedrelationship is stored in the data server 113.

If any one of the above-described conditions is not satisfied, the layerboundary identifying unit 108 determines that there is not any templatethat suits for the concerned A-scan line and does not identify anyretinal layer boundary.

(Step S705) In step S705, the template selection unit 107 determineswhether the total number of the edges counted along the A-scan line isonly one. If there is only one counted edge, the template selection unit107 identifies the interface of the inner limiting membrane as a layerboundary to be identified. Then, the processing proceeds to step S706.

If there is not any counted edge or if the total number of the countededges is three or more, the template selection unit 107 determines thatan appropriate template is not present. Then, the processing proceeds tostep S708.

As described above, if there is not any appropriate template that can beapplied to a detected layer boundary, the template selection unit 107presumes that the layer boundary detection has failed and does notperform identification processing.

As described above, the template selection unit 107 performs layerstructure determination processing along the A-scan line using theentire information of the A-scan line. Therefore, the layer structuredetermination according to the present exemplary embodiment is robustagainst noise and structural change (or modification).

(Step S706) In step S706, the template selection unit 107 selects atemplate that is applicable when only one feature point is present, andacquires the selected template from the storage unit 111.

(Step S707) The layer boundary identifying unit 108 performs layerboundary identification processing along the A-scan line that wasdetermined as positioning in an area where only the inner limitingmembrane is present in an image.

In this case, the layer boundary identifying unit 108 identifies theposition of one feature point as the interface of the inner limitingmembrane with reference to the template information.

The layer boundary identifying unit 108 can also perform the followingprocessing. Example processing that can be performed by the layerboundary identifying unit 108 is described below in detail withreference to FIG. 10. FIG. 10 illustrates an example tomography image ofretinal layers, in which a vertical line A10 represents one of theA-scan lines. The tomography image illustrated in FIG. 10 furtherincludes a profile PSA10 obtained from the Sobel image A and a profilePMD10 obtained from the median image.

Further, a plurality of profiles used in the present step includes areference profile PRE10 that can be derived from the profile obtainedfrom the median image. The reference profile PRE10 is additionallyprepared to identify the boundary of the inner limiting membrane. Thereference profile PRE10 is a typical example of the profile taken alongthe A-scan line extending in an area where the inner limiting membraneand an underlying pseudo-image are present. The layer boundaryidentifying unit 108 determines whether the detected peak coincides withthe inner limiting membrane.

In the present step, the layer boundary identifying unit 108 calculatesan average luminance value in a predetermined range on one side of thepeak as well as in a predetermined range on the other side of the peakwith reference to the profile PSA10 and the profile PMD10. Morespecifically, the layer boundary identifying unit 108 sets twocalculation ranges having a length corresponding to ten pixels(hereinafter, referred to as peak-up B1 and peak-down B2) on both sidesof the peak on the A-scan line.

Next, the layer boundary identifying unit 108 compares the calculatedaverage luminance values to determine a relationship between them in themagnitude while taking the typical example profile into consideration.Only one condition to be satisfied with respect to the average luminancevalues, which can be derived from the reference profile PRE10, ispeak-up B1<peak-down B2.

The layer boundary identifying unit 108 confirms whether the calculatedaverage luminance values coincide with the interface of the innerlimiting membrane by checking whether the calculated average luminancevalues satisfy the above-described condition.

If the above-described condition is satisfied, the layer boundaryidentifying unit 108 identifies the peak as the interface of the innerlimiting membrane. The identified relationship is stored in the dataserver 113.

If the above-described condition is not satisfied, the layer boundaryidentifying unit 108 does not identify any retinal layer boundary alongthe concerned A-scan line.

(Step S708) In step S708, the layer boundary identifying unit 108determines whether the A-scan lines located in an area including atarget layer boundary to be identified have been all selected. In thedetermination in this step, the presence of an appropriate template isnot taken into consideration.

If the processing of the flowchart illustrated in FIG. 7 is completedfor all of the A-scan lines each positioned in the area including thetarget retinal layer boundary to be identified, the processing proceedsto step S406. If it is determined that there is at least one A-scan linethat is not selected (NO in step S708), the processing proceeds to stepS701.

As described above, the layer boundary identifying unit 108 determinesthe type of a layer boundary to be identified according to the number ofedges (i.e., the feature points) detected along the A-scan line atpredetermined intervals in the horizontal direction.

Thus, the layer boundary identifying unit 108 can identify the type ofeach layer boundary according to a change in structure or feature in animage. Further, the layer boundary identifying unit 108 can identify theposition and the type of each layer boundary based on templateinformation according to the number of feature points. Therefore, thelayer boundary identifying unit 108 can identify the type and theposition of each layer boundary according to a change in structure orfeature in an image.

Next, example processing that can be performed by the layer boundaryinterpolation unit 109 in step S406 is described below with reference tothe flowchart illustrated in FIG. 11. The layer boundary interpolationunit 109 performs the above-described interpolation processing for eachlayer boundary type.

The layer boundary interpolation unit 109 sets a search range for anunidentified part of a layer boundary whose remaining part was alreadyidentified and then identifies the position of the layer boundary in thesearch range.

For example, when a target layer boundary to be interpolated is theinterface of the inner limiting membrane, the layer boundaryinterpolation unit 109 identifies the position of the inner limitingmembrane at the A-scan line extending in an area where the innerlimiting membrane was not identified based on positional information ofthe inner limiting membrane already identified by the layer boundaryidentifying unit 108.

(Step S1101) In step S1101, the layer boundary interpolation unit 109selects an A-scan line extending in an area including a target portionof a layer boundary to be identified. In the present exemplaryembodiment, the layer boundary interpolation unit 109 selects the A-scanline extending in an area where the position of a layer boundary was notidentified in the above-described processing performed in step S405. Inthis case, the A-scan line extending in an area where the position of alayer boundary was not identified is the A-scan line to which notemplate is applicable.

(Step S1102) The layer boundary interpolation unit 109 sets a local area(hereinafter, referred to as “neighborhood area”) that surrounds theA-scan line extending in an area where the position of a layer boundarywas not identified. The layer boundary interpolation unit 109 performsinterpolation processing based on coordinate information of the retinallayer boundary identified in the neighborhood area.

The processing to be performed in step S1102 is described below in moredetail with reference to FIG. 12, in which A12 represents the A-scanline extending in the area where no retinal layer boundary wasidentified and R represents the neighborhood area surrounding the A-scanline A12 positioned at the center thereof. The layer boundaryinterpolation unit 109 performs range setting in such a way as toinvolve a predetermined number of A-scan lines in the neighborhood area.

The above-described neighborhood area is a rectangular area having eachside comparable to nine A-scan lines (i.e., a square of 9×9 A-scanlines), in which the target A-scan line (i.e., the A-scan line extendingin an area including a target layer boundary to be identified) ispositioned at the center thereof.

In the neighborhood area set around the target A-scan line, if the totalnumber of the A-scan lines along which the retinal layer boundary hasalready been identified is less than a predetermined number (NO in stepS1102), the layer boundary interpolation unit 109 determines that theinterpolation processing to be performed will become unreliable. In thiscase, the processing proceeds to step S1107. Namely, the layer boundaryinterpolation unit 109 skips the interpolation processing.

If the total number of the A-scan lines along which the retinal layerboundary has already been identified is greater than the predeterminednumber (YES in step S1102), then in step S1103, the layer boundaryinterpolation unit 109 performs the interpolation processing.

If the total number of the identification completed A-scan lines is lessthan the predetermined number, the layer boundary interpolation unit 109repeats the processing of steps S1101 through S1107 to successivelyidentify retinal layer boundaries located in the neighborhood area. Whenthe total number of the identification completed A-scan lines exceedsthe predetermined number, the layer boundary interpolation unit 109starts the interpolation processing.

In the present exemplary embodiment, the above-described predeterminednumber is equal to a half of the total number of the A-scan lines thatare present in the neighborhood area, i.e., 40.

(Step S1103) In step S1103, the layer boundary interpolation unit 109calculates a feature quantity required to identify a retinal layerboundary about the A-scan line extending in an area where the positionof the retinal layer boundary was not identified. The layer boundaryinterpolation unit 109 calculates the feature quantity based oninformation relating to the A-scan line along which the retinal layerboundary has already been identified.

In the present exemplary embodiment, the layer boundary interpolationunit 109 sets a local area (hereinafter, referred to as “neighborhoodarea”) that surrounds the A-scan line along which the position of theretinal layer boundary was not identified. The layer boundaryinterpolation unit 109 calculates a reference position based on theretinal layer boundary already identified in the neighborhood area. Asillustrated in FIG. 12, the A-scan line along which the position of theretinal layer boundary was not identified is located at the center ofthe neighborhood area set by the layer boundary interpolation unit 109.

The layer boundary interpolation unit 109 obtains an averagez-coordinate value of each retinal layer boundary identified in theneighborhood area and designates the obtained z-coordinate position as areference position.

The determination condition to be satisfied in executing the calculationof the reference position is similar to that used in step S402. Morespecifically, the layer boundary interpolation unit 109 performs thereference position calculation processing by checking whether the totalnumber of the A-scan lines along which the retinal layer boundary hasalready been identified in the neighborhood area is greater than orsmaller than a predetermined number.

If the reference position calculation processing is completed, the layerboundary interpolation unit 109 determines an image area that can beregarded as including the layer boundary based on the calculatedreference position of each layer as a search range. The layer boundaryinterpolation unit 109 identifies the retinal layer boundary thatcorresponds to the A-scan line in the above-described search range.

The layer boundary interpolation unit 109 sets a predetermined range inthe depth direction, as a search range for the retinal layer boundary,using the average z-coordinate value of each retinal layer that has beencalculated as the reference position. For example, the predeterminedrange in the depth direction set by the layer boundary interpolationunit 109 includes an upper range corresponding to five pixels set on theupper side of the average z-coordinate value and a lower rangecorresponding to five pixels set on the lower side of the averagez-coordinate value.

As described above, the layer boundary interpolation unit 109 refers toa depth directional position of a layer boundary whose position hasalready been identified by the layer boundary identifying unit 108 toset a search range for a portion of the layer boundary whose positionwas not identified the layer boundary identifying unit 108.

The search range setting is not limited to the above-described exampleand any other appropriate setting can be employed. For example, if animage includes a small amount of noises, the layer boundaryidentification processing can be performed accurately by setting agreater search range.

Further, if the structure of a layer is comparatively simple and flat,it is unnecessary to set a greater search range. Therefore, it may beuseful to change the search range with reference to structuralinformation of peripheral layers. Further, when the isotropy of noisesis taken into consideration, it may be useful to locally separate noisesfrom signal components by extracting noise components from the entireimage or along an A-scan line to be processed.

(Step S1104) In step S1104, the layer boundary interpolation unit 109searches for a peak area along the profile of the Sobel image A withinthe search range. In this case, the peak area is an area in the vicinityof a local maximum value that is equal to or greater than apredetermined value in the profile of the Sobel image A.

The peak area corresponds to an area where the luminance change in atomography image exceeds a predetermined threshold value. Theabove-described predetermined threshold value corresponds to a secondthreshold value according to the present invention.

As the layer boundary interpolation unit 109 can set a narrower searchrange to effectively perform peak search processing, the layer boundaryinterpolation unit 109 can find a smaller peak area with a thresholdvalue that is smaller than the first threshold value set by thedetection unit 106 in step S404.

Further, as the layer boundary interpolation unit 109 can set a searchrange based on an average value of the depth directional position of thelayer boundary already identified in the neighborhood area, thepossibility that the layer boundary is present in the search range setby the layer boundary interpolation unit 109 is higher.

(Step S1105) The layer boundary interpolation unit 109 determineswhether the peak area is present. If the peak area is present, the layerboundary interpolation unit 109 determines that the position of thetarget layer boundary to be interpolated can be identified. Therefore,the processing proceeds to step S1106 in which the layer boundaryinterpolation unit 109 continues the interpolation processing.

If no peak area is present, the layer boundary interpolation unit 109determines that the reliability of the interpolation processing ifperformed becomes lower. Therefore, the layer boundary interpolationunit 109 skips the interpolation processing. The processing proceeds tostep S1107.

If there is not any peak area detected in step S1105, the layer boundaryinterpolation unit 109 may identify a position where the edge componentof the Sobel image A becomes largest in the search range as a retinallayer boundary.

(Step S1106) The layer boundary interpolation unit 109 identifies theposition of the peak area having a largest luminance change as theposition of the target boundary.

(Step S1107) The layer boundary interpolation unit 109 determineswhether the above-described layer boundary identification processing hasbeen completed for all of the A-scan lines located in the area where thetarget layer boundary to be identified is present.

If the layer boundary interpolation unit 109 determines that there is atleast one A-scan line not subjected to the above-described layerboundary identification processing, the processing returns to step S1101in which the layer boundary interpolation unit 109 repeats theinterpolation processing.

In this case, the layer boundary interpolation unit 109 newly sets asearch range for a layer portion whose position was not identified,based on the depth directional position of a layer boundary whoseposition has already been identified by the layer boundary identifyingunit 108 or the layer boundary interpolation unit 109.

Then, the layer boundary interpolation unit 109 further identifies theposition of the layer boundary portion whose position was notidentified, based on a luminance change in the depth direction withinthe newly set search range. The layer boundary interpolation unit 109repetitively performs the above-described processing to successivelyidentify an unidentified portion of the layer boundary, with referenceto the closest identified portion of the layer boundary.

If the above-described layer boundary identification processing iscompleted for all target A-scan lines, the layer boundary interpolationunit 109 terminates the processing of the flowchart illustrated in FIG.11. Alternatively, the layer boundary interpolation unit 109 mayterminate the processing of the flowchart illustrated in FIG. 11 if theprocessing time exceeds a predetermined time or when the total number ofthe repetitively performed loop processing exceeds a predeterminedvalue.

Moreover, the total number of the A-scan lines having been subjected tothe above-described layer boundary identification processing may notchange even after the layer boundary interpolation unit 109 hasperformed the loop processing of steps S1101 to S1107 a predeterminednumber of times. In such a case, the layer boundary interpolation unit109 can forcibly terminate the processing of the flowchart illustratedin FIG. 11.

As described above, the layer boundary interpolation unit 109 determinesa search range based on the depth directional position of the alreadyidentified layer boundary and identifies the position of an unidentifiedportion of the layer boundary based on a luminance change in the searchrange having been set.

Thus, the present exemplary embodiment can improve the accuracy of thelayer boundary identification processing compared to a case where thelayer boundary identification processing is performed based on aluminance change in the entire range of the A-scan line.

Further, the layer boundary interpolation unit 109 can set a narrowersearch range and can set a smaller threshold value to be used to searchfor a peak. Thus, the layer boundary interpolation unit 109 can identifythe position of an unidentified portion of the layer boundary based on asmaller luminance change.

As described above, the image processing apparatus according to thepresent exemplary embodiment calculates a luminance value and a boundaryposition of each retinal layer in the neighborhood area, in theprocessing to be performed to identify a retinal layer boundary includedin a tomography image of an eye to be examined.

Then, the image processing apparatus according to the present exemplaryembodiment sets conditions to be satisfied to perform retinal layerboundary identification processing based on the obtained information.Thus, the image processing apparatus 101 can accurately perform theretinal layer boundary identification processing even when the luminancevalue or the layer structure of a target retinal layer is changed.

For example, according to the OCT, the profile of the same retinal layermay change greatly depending on a portion to be examined. Even when sucha change occurs in the luminance value, the image processing apparatusaccording to the present exemplary embodiment can set appropriateretinal layer boundary identification conditions for a target layerportion to be identified referring to the information of each retinallayer obtainable in the neighborhood area and can accurately identifythe position of the retinal layer boundary.

Further, the image processing apparatus according to the presentexemplary embodiment can set appropriate retinal layer boundaryidentification conditions from an input image, without performing manualadjustment of the threshold value, while taking individual differencesand machine model differences in the tendency of the luminance valueinto consideration.

Accordingly, the image processing apparatus according to the presentexemplary embodiment can eliminate errors in identifying the type ofeach layer boundary even when different types of later structures appearin an optical coherence tomography image of an eye to be examined.

In a second exemplary embodiment, the present invention is applied toidentification of the interface of the nerve fiber layer, the interfaceof the inner plexiform layer, and the interface of the outer plexiformlayer.

Further, the present exemplary embodiment includes processing to beperformed to determine whether the profile of an image coincides withtemplate information when the template information is selected. Thetemplate selection unit 107 can serve as the above-describeddetermination unit. It may also be desired to provide a circuit capableof functionally operable as the determination unit.

In the present exemplary embodiment, it is presumed that the interfaceof the inner limiting membrane and the interface between inner and outersegments of the photoreceptors are identified beforehand. The presentexemplary embodiment includes preparing a reference profile thatrepresents a layer structure for each of the nerve fiber layer, theinner plexiform layer, and the outer plexiform layer. The presentexemplary embodiment further includes selecting an optimum referenceprofile to be used to identify each retinal layer boundary withreference to the number of feature points obtained from the profile ofthe Sobel image B.

The present exemplary embodiment includes processing to be performed todetermine whether the selected template is appropriate with reference toluminance values obtained in upper and lower areas of a luminance changepeak position. Respective retinal layers are different from each otherin reflectance. Therefore, accurately identifying each layer boundaryand the type of the detected layer are feasible based on the luminancevalues of the above-described layer areas. An image processing systemaccording to the present exemplary embodiment is similar to thatdescribed in the first exemplary embodiment and therefore itsdescription is not repeated.

An example flow of processing that can be performed by the imageprocessing apparatus 101 according to the present exemplary embodimentis described below with reference to a flowchart illustrated in FIG. 13.The processing in FIG. 13 includes a portion similar to the processingdescribed in the first exemplary embodiment and therefore itsdescription is not repeated.

(Step S1304) In step S1304, the image processing apparatus 101identifies the interface of the inner limiting membrane and theinterface between inner and outer segments of the photoreceptors. Theimage processing apparatus 101 can perform the above-describedidentification processing using the method described in the firstexemplary embodiment or using another method.

For example, the image processing apparatus 101 acquires a tomographyimage in which the position of the above-described layer boundary isidentified beforehand. Then, the image processing apparatus 101 canperform the above-described identification processing based on theidentified position data.

The image processing apparatus 101 identifies the interface of the nervefiber layer, the interface of the inner plexiform layer, and theinterface of the outer plexiform layer based on the above-describedpositions of the interface of the inner limiting membrane and theinterface between inner and outer segments of the photoreceptors.

(Step S1306) In step S1306, the template selection unit 107 selectstemplate information and the layer boundary identifying unit 108performs layer boundary identification processing. The present exemplaryembodiment is different from the first exemplary embodiment inperforming processing to determine whether the selected templateinformation matches with the profile of the tomography image, after thetemplate selection processing is completed, as described below in moredetail.

(Step S1307) In step S1307, the image processing apparatus 101 performsinterpolation processing on a layer boundary that was not identified instep S1306 to interpolate the position of the unidentified layerboundary. The processing to be performed in step S1307 is different fromthe processing described in the first exemplary embodiment inidentifying a peak position of a peak area that is closest to thecalculated average Z-coordinate, among peak areas discovered in thesearch range, as the position of the layer boundary, as described belowin more detail.

An example flow of the processing to be performed by the templateselection unit 107 and the layer boundary identifying unit 108 in stepS1306 is described below with reference to the flowchart of FIG. 14. Theprocessing in FIG. 14 includes a portion similar to the processingdescribed in the first exemplary embodiment and therefore itsdescription is not repeated.

(Step S1402) Processing to be performed in step S1402 contains all ofthe processing performed in steps S702, S703, S705, and S706 describedin the first exemplary embodiment. The processing to be performed instep S1402 is described below in more detail with reference to FIG. 15.FIG. 15 illustrates an example tomography image of retinal layers, inwhich vertical lines A151 and A152 represent two A-scan lines. Thetomography image illustrated in FIG. 15 further includes profiles PSB151and PSB152 obtained from the Sobel image B.

The depth directional range of the profile to be used in the layerstructure determination processing has an upper edge and a lower edgethat correspond to the interface of the inner limiting membrane and theinterface between inner and outer segments of the photoreceptors thatare identified beforehand, respectively. Therefore, the layer structuredetermination processing is performed only in the range indicated bysolid lines of the profiles PSB151 and PSB152 illustrated in FIG. 15.

To estimate each retinal layer, the template selection unit 107 countsthe number of peaks appearing along the profile in the range extendingfrom the inner limiting membrane to the interface between inner andouter segments of the photoreceptors. The nerve fiber layer may not beincluded in an image if the position of a selected A-scan line isinappropriate (see the profile PSB152 illustrated in FIG. 15).

Considering the above-described situation, the template selection unit107 counts the number of peaks along the profile of the Sobel image B ineach A-scan line and, if the number of detected peaks is three, thetemplate selection unit 107 determines that there is a higherpossibility that the nerve fiber layer, the inner plexiform layer, andthe outer plexiform layer are present along the target A-scan line.

The above-described each layer is a layer boundary or a candidate of thelayer that exists at a position corresponding to the A-scan line(processing target) and the peak position is a candidate of the positionof the above-described each layer. Thus, in this case, the templateselection unit 107 selects a reference profile employable when thenumber of peaks is three.

If the number of detected peaks is two, the template selection unit 107determines that there is a higher possibility that the inner plexiformlayer and the outer plexiform layer are present along the target A-scanline. The above-described each layer is a candidate of the layer thatexists at a position corresponding to the A-scan line (processingtarget) and the peak position is a candidate of the position of theabove-described each layer. Thus, in this case, the template selectionunit 107 selects a reference profile employable when the number of peaksis two.

In the present exemplary embodiment, the image used in theabove-described layer boundary identification processing is the Sobelimage B. The above-described Sobel image B is an image that can beobtained by extracting each lower-side edge at which the luminance valuechanges from a larger side to a lower side in the depth direction.

(Step S1403) In step S1403, the template selection unit 107 determineswhether the reference profile has been selected for the A-scan line(i.e., the processing target). If it is determined that the referenceprofile has been selected (YES in step S1403), the template selectionunit 107 can perform layer boundary identification processing.Therefore, the processing proceeds to step S1404.

If no reference profile has been selected (NO in step S1403), thetemplate selection unit 107 determines that the layer boundaryidentification processing is unfeasible due to the presence of noise.Therefore, the processing proceeds to step S1408 in which the templateselection unit 107 starts template selection processing for the nextA-scan line.

(Step S1404) The determination unit determines whether the templateselected in step S1403 matches the A-scan line to be processed. To thisend, in step S1404, the determination unit calculates luminance valuesof respective feature points.

(Step S1405) In step S1405, the determination unit compares thecalculated luminance values of respective feature points in themagnitude with luminance information of the reference profile (i.e., thetemplate information).

(Step S1406) In step S1406, the determination unit determines whetherthe template information matches the A-scan line to be processed basedon the above-described comparison result. If the template informationmatches the A-scan line to be processed (YES in step S1406), thedetermination unit determines that the layer boundary identificationprocessing is feasible. Therefore, the processing proceeds to stepS1407.

If the template information does not match the A-scan line to beprocessed (NO in step S1406), the determination unit determines that thetemplate determination is unfeasible. Therefore, the image processingapparatus 101 skips the identification processing. The processingproceeds to step S1408.

The processing to be performed in step S1407 and step S1408 is similarto the processing described in the first exemplary embodiment andtherefore its description is not repeated. Hereinafter, the processingto be performed in steps S1404 to S1406 is described below in moredetail.

Example processing to be performed when the number of detected featurepoints is three is described below in more detail with reference to FIG.16. FIG. 16 illustrates an example tomography image of retinal layers,in which a vertical line A16 represents one of the A-scan lines. Thetomography image illustrated in FIG. 16 further includes a profile PSB16obtained from the Sobel image B and a profile PMD16 obtained from themedian image.

Further, a plurality of profiles used in the present step includes areference profile PRE16 that can be derived from the profile obtainedfrom the median image. The reference profile PRE16 is additionallyprepared to identify each boundary of the target three layers (i.e., thenerve fiber layer, the inner plexiform layer, and the outer plexiformlayer).

The reference profile PRE16 is a typical example of the profile takenalong the A-scan line extending in an area where the nerve fiber layer,the inner plexiform layer, and the outer plexiform layer are present.

In the present exemplary embodiment, the typical example indicates atendency of the luminance between respective layers along the A-scanline extending in an area where the target three layers (i.e., the nervefiber layer, the inner plexiform layer, and the outer plexiform layer)are present, in which no noise is present.

For example, it is experimentally known that the luminance value of thenerve fiber layer tends to be higher than those of the inner plexiformlayer and the outer plexiform layer. Therefore, the prepared profilePRE16 possesses the above-described tendency in the luminance features.

The reference profile PRE16 is not limited to the above-describedexample. Any other profile can be employed if the luminance tendency ofeach retinal layer and a positional relationship between retinal layerscan be identified based on the employed profile.

The reference profile PRE16 does not reflect the thickness of eachretinal layer, although the thickness is generally variable depending onthe position of the A-scan line. Therefore, the reference profile PRE16is employable for any A-scan line extending in an area where the targetthree layers (i.e., the nerve fiber layer, the inner plexiform layer,and the outer plexiform layer) are present.

The determination unit uses the reference profile PRE16 to determinewhether the detected three peaks coincide with the above-described threelayers (i.e., the nerve fiber layer, the inner plexiform layer, and theouter plexiform layer).

First, the determination unit calculates an average luminance valuebetween respective peaks appearing on the profile PSB16. Thedetermination unit uses, as a luminance value, a value of the profilePMD16 corresponding to the position of the profile PSB16. Thedetermination unit refers to three peaks of the profile PSB16 as a firstpeak, a second peak, and a three peak, respectively, which arepositioned in this order from the shallow side.

The range in which the determination unit calculates the averageluminance value is a range extending from the inner limiting membrane tothe first peak, a range extending from the first peak to the secondpeak, a range extending from the second peak to the third peak, and arange extending from the third peak to the interface between inner andouter segments of the photoreceptors.

The above-described ranges are successively referred to as peak-to-peakA1, peak-to-peak A2, peak-to-peak A3, and peak-to-peak A4. Further, therange “peak-to-peak A3” (i.e., the range extending from the second peakto the third peak) includes two equally divided ranges. One of theabove-described two divided ranges, which is adjacent to the secondpeak, is referred to as “peak-to-peak A31.” The other of theabove-described two divided ranges, which is adjacent to the third peak,is referred to as “peak-to-peak A32.” The determination unit calculatesaverage luminance values in the above-described ranges using the profilePMD16.

Next, the determination unit compares the calculated average luminancevalues to determine a relationship between them in the magnitude whiletaking the reference profile PRE16 into consideration. Conditions to besatisfied with respect to the average luminance values, which can bederived from the reference profile PRE16, are peak-to-peakA1>peak-to-peak A2, peak-to-peak A1>peak-to-peak A3, peak-to-peakA3>peak-to-peak A4, and peak-to-peak A31<peak-to-peak A32.

The determination unit confirms whether the calculated average luminancevalues coincide with the target three layers (i.e., the nerve fiberlayer, the inner plexiform layer, and the outer plexiform layer) bychecking whether the calculated average luminance values satisfy theabove-described conditions.

If the above-described conditions are all satisfied, the determinationunit identifies the first peak, the second peak, and the third peak asboundaries of the nerve fiber layer, the inner plexiform layer, and theouter plexiform layer. The identified relationship is stored in the dataserver 113.

If at least one of the above-described conditions is not satisfied, thedetermination unit does not identify any retinal layer boundary alongthe concerned A-scan line.

Example processing to be performed when the number of detected featurepoints is two is described below in more detail with reference to FIG.17. FIG. 17 illustrates an example tomography image of retinal layers,in which a vertical line A17 represents one of the A-scan lines. Thetomography image illustrated in FIG. 17 further includes a profile PSB17obtained from the Sobel image B and a profile PMD17 obtained from themedian image.

Further, a plurality of profiles used in the present step includes areference profile PRE17 that can be derived from the profile obtainedfrom the median image. The reference profile PRE17 is additionallyprepared to identify each boundary of the target two layers (i.e., theinner plexiform layer and the outer plexiform layer).

The reference profile PRE17 is a typical example of the profile takenalong the A-scan line extending in an area where the inner plexiformlayer and the outer plexiform layer are present.

Similar to the description in step S1402, the typical example indicatesa tendency of the luminance between respective layers along the A-scanline extending in an area where the target two layers (i.e., the innerplexiform layer and the outer plexiform layer) are present, in which nonoise is present.

Therefore, the reference profile PRE17 is employable for any A-scan lineextending in an area where the target two layers are present. Thedetermination unit uses the reference profile PRE17 to determine whetherthe detected two peaks coincide with the inner plexiform layer and theouter plexiform layer.

In the present step, the determination unit calculates an averageluminance value between respective peaks using the profile PSB17 and theprofile PMD17. The determination unit refers to two peaks of the profilePSB17 as a first peak and a second peak, respectively, which arepositioned in this order from the shallow side.

The range in which the determination unit calculates the averageluminance value is a range extending from the inner limiting membrane tothe first peak, a range extending from the first peak to the secondpeak, and a range extending from the second peak to the interfacebetween inner and outer segments of the photoreceptors.

The above-described ranges are successively referred to as peak-to-peakB1, peak-to-peak B2, and peak-to-peak B3. Further, the range“peak-to-peak B2” (i.e., the range extending from the first peak to thesecond peak) includes two equally divided ranges. One of theabove-described two divided ranges, which is adjacent to the first peak,is referred to as “peak-to-peak B21.” The other of the above-describedtwo divided ranges, which is adjacent to the second peak, is referred toas “peak-to-peak B22.” The determination unit calculates averageluminance values in the above-described ranges using the profile PMD17.

Next, the determination unit compares the calculated average luminancevalues to determine a relationship between them in the magnitude whiletaking the typical example profile into consideration. Conditions to besatisfied with respect to the average luminance values, which can bederived from the reference profile PRE17, are peak-to-peakB1>peak-to-peak B2, peak-to-peak B2>peak-to-peak B3, and peak-to-peakB21<peak-to-peak B22.

The determination unit confirms whether the calculated average luminancevalues coincide with the target two layers (i.e., the inner plexiformlayer and the outer plexiform layer) by checking whether the calculatedaverage luminance values satisfy the above-described conditions.

If the above-described conditions are all satisfied, the determinationunit identifies the first peak and the second peak as boundaries of theinner plexiform layer and the outer plexiform layer. The identifiedrelationship is stored in the data server 113.

If at least one of the above-described conditions is not satisfied, thedetermination unit does not identify any retinal layer boundary alongthe concerned A-scan line.

As described above, determining whether a template selected according tothe total number of detected feature points coincides with a profiletaken along a target A-scan line to be processed is useful to eliminateerrors in the layer boundary identification processing that may bederived from an error in template selection. Further, using luminancevalues of inter-boundary areas in appropriately determining the templateinformation is useful to eliminate errors in the layer boundaryidentification processing.

An image processing system according to a third exemplary embodimentincludes an algorithm switching unit configured to select an optimumalgorithm based on pattern matching between profiles, without countingthe number of peaks appearing along a profile obtained from the Sobelimage.

More specifically, an image processing apparatus adjusts a positionalrelationship between a target profile of a median image to be processedand each reference profile and calculates a distance betweencorresponding signals. Then, the image processing apparatus selects anoptimum algorithm to be used in retinal layer boundary identificationwith reference to the layer structure of a reference profile which issmallest in the cumulative value of the calculated distance. In thepresent exemplary embodiment, an example algorithm switching to beperformed by the image processing apparatus to identify the nerve fiberlayer, the inner plexiform layer, and the outer plexiform layer isdescribed below.

An example flow of processing that can be performed by the templateselection unit 107 and the layer boundary identifying unit 108 accordingto the present exemplary embodiment is described below with reference tothe flowchart illustrated in FIG. 18. The flowchart in FIG. 18 includesa portion similar to the processing described in the second exemplaryembodiment with reference to the flowchart illustrated in FIG. 14 andtherefore its description is not repeated.

(Step S1802) In step S1802, the template selection unit 107 calculates asimilarity between the target profile and each of all reference profilesalong each A-scan line. Then, the template selection unit 107 selects analgorithm based on the calculated similarity. In the present exemplaryembodiment, the template selection unit 107 can use a pattern matchingmethod to calculate the above-described similarity based on a comparisonbetween profiles. An example calculation method is described below.

Example processing is described below in more detail with reference toFIG. 19. A tomography image illustrated in FIG. 19 includes one A-scanline A19 and a profile PMD190 obtained from the median image. Thetomography image illustrated in FIG. 19 further includes a referenceprofile PRE191 that can be obtained when three layers of the nerve fiberlayer, the inner plexiform layer, and the outer plexiform layer arepresent. The tomography image illustrated in FIG. 19 further includes areference profile PRE192 that can be obtained when two layers of innerplexiform layer and the outer plexiform layer are present.

First, the template selection unit 107 adjusts the positionalrelationship between a median image profile taken along the targetA-scan line to be processed and each reference profile. The templateselection unit 107 can perform the above-described positioningprocessing with reference to the positions of the inner limitingmembrane and the interface between inner and outer segments of thephotoreceptors, which have already been identified in step S1406 or itspreceding step.

The template selection unit 107 expands or contracts the referenceprofile PRE191 and the reference profile PRE192 in such a way as toadjust the positions of the inner limiting membrane and the interfacebetween inner and outer segments of the photoreceptors withcorresponding positions of the profile PMD190.

Then, the template selection unit 107 overlaps each of the referenceprofile PRE191 and the profile PRE192 with the profile PMD190, asindicated by profiles PMT201 and PMT202 illustrated in FIG. 20. Further,the template selection unit 107 calculates the distance between theoverlapped profiles at each corresponding point.

In the present exemplary embodiment, the distance calculated by thetemplate selection unit 107 is a difference in the x-coordinatedirection between the overlapped profiles measured at the samez-coordinate position. The template selection unit 107 obtains acumulative value of the above-described distance for each referenceprofile.

(Step S1803) In step S1803, the template selection unit 107 selects anoptimum template with reference to the calculated similarity. In thepresent exemplary embodiment, the template selection unit 107 selects atemplate that fits the structure of the reference profile having alowest cumulative value (i.e., highest similarity).

The selected template information is associated with position/typeinformation of a layer boundary. Therefore, the template selection unit107 can determine the type of a layer boundary (processing target) to beidentified along the A-scan line. In the present exemplary embodiment,if it is determined that there is a higher similarity between theprofile PMD190 and the reference profile PRE191, the template selectionunit 107 determines that the nerve fiber layer, the inner plexiformlayer, and the outer plexiform layer are present along the A-scan line.Then, the processing proceeds to step S1804.

As described above, the image processing apparatus according to thepresent exemplary embodiment selects an optimum template based on thepattern matching processing and identifies the position and the type ofeach layer boundary based on template information. Therefore, the imageprocessing apparatus according to the present exemplary embodiment canidentify the type and the position of each layer boundary according to achange in the structure or the feature of an image.

If the calculated similarity is high when the template selection isperformed based on the pattern matching as described in the presentexemplary embodiment, it is unnecessary to perform the adaptabilitydetermination processing described in the second exemplary embodiment.In general, when the calculated similarity is high, it can be regardedthat the compared profiles substantially coincide with each other.

In this case, the processing can be simplified because the imageprocessing apparatus is not required to perform the above-describedselection processing based on the number of detected feature points andcan skip the subsequent determination processing.

On the other hand, in the second exemplary embodiment, it may be usefulto perform the pattern matching processing according to the presentexemplary embodiment as adaptability determination processing to beperformed by the determination unit. In this case, the determinationunit performs the determination processing by checking whether thecalculated similarity exceeds a predetermined threshold value.

Further, the template to be used in the present exemplary embodiment canbe a reference profile itself that can be created with reference to theaverage (or median) of a profile of a tomography image of layerboundaries having been identified beforehand.

A fourth exemplary embodiment is characterized in that, in theinterpolation processing to be performed by the layer boundaryinterpolation unit 109, a position that is closest to an average depthdirectional value of the identification completed layer boundary, amongluminance change peaks in the search range, is identified as theposition of the layer boundary.

The image processing apparatus according to the present exemplaryembodiment performs interpolation processing on the interface of thenerve fiber layer, the interface of the inner plexiform layer, and theinterface of the outer plexiform layer in the following manner.

FIG. 21 is a flowchart illustrating example retinal layer boundaryinterpolation processing according to the present exemplary embodiment.The flowchart illustrated in FIG. 21 includes a portion similar to theprocessing described in the first exemplary embodiment with reference tothe flowchart illustrated in FIG. 11 and therefore its description isnot repeated.

(Step S2103) In step S2103, the layer boundary interpolation unit 109calculates a feature quantity required to identify a retinal layerboundary about the A-scan line extending in an area where the positionof the retinal layer boundary was not identified. The layer boundaryinterpolation unit 109 calculates the feature quantity based oninformation relating to the A-scan line along which the retinal layerboundary has already been identified.

In the present exemplary embodiment, the layer boundary interpolationunit 109 sets a local area (hereinafter, referred to as “neighborhoodarea”) that surrounds the A-scan line along which the position of theretinal layer boundary was not identified. The layer boundaryinterpolation unit 109 calculates a feature quantity based on theretinal layer boundary already identified in the neighborhood area.

Similar to step S1103, in the feature quantity calculation, the A-scanline along which the position of the retinal layer boundary was notidentified is located at the center of the neighborhood area set by thelayer boundary interpolation unit 109, as illustrated in FIG. 12. Thelayer boundary interpolation unit 109 obtains an average z-coordinatevalue of each retinal layer boundary identified in the neighborhood areaand designates the obtained z-coordinate position as a feature quantity.

Similar to step S1102, the layer boundary interpolation unit 109determines whether to perform the feature quantity calculationprocessing by checking whether the total number of the A-scan linesalong which the retinal layer boundary has already been identified inthe neighborhood area is greater than or smaller than a predeterminednumber.

The layer boundary interpolation unit 109 sets a search range for eachretinal layer boundary referring to the average z-coordinate value ofthe retinal layer calculated as the above-described feature quantity.The search range set by the layer boundary interpolation unit 109 is apredetermined range including the average z-coordinate value positionedat the center thereof. In the present exemplary embodiment, thepredetermined range is composed of five pixels positioned on the frontside of the average z-coordinate value and five pixels positioned on therear side of the average z-coordinate value.

(Step S2106) The layer boundary interpolation unit 109 searches for apeak appearing on the profile of the Sobel image B in the search range.If at least one peak is present, the layer boundary interpolation unit109 identifies a largest peak that is equal to or greater than apredetermined threshold value as a retinal layer boundary. If two ormore peaks are present, the layer boundary interpolation unit 109identifies a peak whose position is closest to the calculated averageZ-coordinate value as the position of a target layer boundary.

More specifically, two or more layer boundaries may be included in asearch range set to identify a layer boundary. In such a case, the layerboundary interpolation unit 109 selects a peak whose position is closestto an average depth value of the already identified layer boundary inthe neighborhood area. Thus, each layer boundary can be accuratelyidentified.

FIG. 22 illustrates an example profile PSB220 of the Sobel image B,taken along an A-scan line extending in an area where three layers ofthe nerve fiber layer, the inner plexiform layer, and the outerplexiform layer are present. The profile PSB220 illustrated in FIG. 22includes four peaks because one of the peaks is a noise. In this case,it is difficult to identify each retinal layer boundary depending on themagnitude of each peak.

However, it is feasible to accurately identify each retinal layerboundary by using a search range CN of the nerve fiber layer boundary, asearch range CI of the inner plexiform layer boundary, and a searchrange CO of the outer plexiform layer boundary, which can be set in stepS2104 based on the average z-coordinate value calculated in theneighborhood area.

As described above, even when a plurality of peak positions arediscovered, the retinal layer boundary identification processing can beaccurately performed by identifying a peak position closest to analready identified average z-coordinate value as the position of atarget layer boundary.

Another Exemplary Embodiment

The image processing apparatus described in each exemplary embodiment isa mere example that can realize the present invention. However, thepresent invention is not limited to the image processing apparatus.Further, a comparable software configuration is employable to realizethe image processing apparatus 101 described in each of the first tofourth exemplary embodiments.

In this case, a storage medium storing a program that causes a computerto execute the processing of the image processing apparatus 101according to each of the above-described exemplary embodiments can besupplied to a system or an apparatus. Then, a computer (or a CPU or amicro-processing unit (MPU)) installed in the system or the apparatuscan read a program code from the storage medium and execute the readprogram so as to realize the present invention. In this case, theprogram code itself read from the storage medium realizes the functionsof the above-described exemplary embodiments. The storage medium storingthe program code constitutes the present invention.

In the above-described exemplary embodiments, the image processingapparatus performs analysis processing on three-dimensional image datacomposed of a plurality of tomography images to be acquired. However, itmay also be useful that the image processing apparatus selects atwo-dimensional tomography image to be concerned from thethree-dimensional image data and performs processing on the selectedtomography image.

For example, it may be useful that the image processing apparatusperforms processing on a tomography image including a specific portion(e.g., central pit) of the ocular fundus. In this case, each detectedlayer boundary can be obtained as two-dimensional data that reflect theabove-described cross section.

In the above-described exemplary embodiments, the image processingapparatus identifies each layer boundary. However, the image processingapparatus may identify the type and the position of each layer usingtemplate information. In this case, the image processing apparatusrequires, as the template information, information relating to themagnitude (or luminance value) of a luminance change at the edgecorresponding to the position of each layer boundary, or luminancevalues or its relationship in the magnitude between respective layerareas.

For example, macula, optic disc, retinal area including a blood vessel,and retinal area including no blood vessel can be regarded as templateinformation.

Further, the image processing apparatus can identify a layer structureof the macula or the optic disc based on the total number of layerboundaries and a luminance value of each layer area. For example, theimage processing apparatus can determine whether a target layer is themacula, the optic disc, or another area, such as an area where a bloodvessel is present, an area where a leucoma is present, or an area whereretina detachment is present.

Further, the image processing apparatus can determine whether a retinallayer structure includes a pseudo-image of a blood vessel or a leucomaas a feature that can be recognized in a tomography image.

The layer boundary interpolation processing according to theabove-described exemplary embodiment can be performed independently fromthe above-described layer boundary identification processing to beperformed based on template information. For example, theabove-described layer boundary interpolation processing can be employedin a case where a layer boundary is partly identified using anothermethod.

The tomography image to be used in the above-described exemplaryembodiment is not limited to a tomography image of retinal layers andcan be a tomography image of an anterior ocular portion. The anteriorocular portion has a multilayered structure that is composed of acornea, a lens, and a vitreous body, which are sequentially disposed inthe incident direction of signal light, i.e., in the depth direction.

Therefore, the present invention can be applied to a tomography image ofthe anterior ocular portion. In this case, the storage unit 111 stores aplurality of templates that represent a plurality of structures in theprofile of the tomography image of the anterior ocular portion.

The image processing system 100 described in the above-describedexemplary embodiment is an example to which the present invention can beapplied. The optical coherence tomography imaging apparatus can beconfigured to have the above-described functions of the image processingapparatus 101.

Further, in the above-described exemplary embodiments, the exampletomography image is a tomography image obtained by the optical coherencetomography imaging apparatus. The present invention can be applied to atomography image obtained by an ultrasonic tomography imaging apparatusor a comparable apparatus capable of imaging an internal structure of atarget to be captured.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Application No.2010-064754 filed Mar. 19, 2010, which is hereby incorporated byreference herein in its entirety.

The invention claimed is:
 1. An image processing apparatus comprising: a first identifying unit configured to identify the position of at least a part of a layer boundary based on a tomography image of a target eye to be captured; a setting unit configured to set a search range for a portion whose position has not been identified by the first identifying unit based on a depth directional position of the layer boundary whose position has been identified by the first identifying unit; and a second identifying unit configured to identify the position of a layer boundary portion whose position has not been identified by the first identifying unit based on a luminance value in a position where the layer boundary identified by the first identifying unit does not exist in the search range having been set, wherein the layer boundary is any one of an interface of an inner limiting membrane, an interface of a nerve fiber layer, an interface of an inner plexiform layer, an interface of an outer plexiform layer, an interface between inner and outer segments of the photoreceptors, or an interface of a retinal pigment epithelium.
 2. The image processing apparatus according to claim 1, wherein the first identifying unit is configured to identify the position of the layer boundary and a type of the layer boundary based on the number of edges disposed in the depth direction in the tomography image.
 3. The image processing apparatus according to claim 1, wherein the setting unit is configured to set, as the search range for the layer boundary, a range including a reference position that is defined based on a depth position of the layer boundary whose position has been identified.
 4. The image processing apparatus according to claim 3, wherein the second identifying unit is configured to identify, as the position of the layer boundary portion whose position has not been identified, the position of a candidate closest to the reference position, if there are a plurality of candidates for the position of the layer boundary portion to be determined based on a luminance change value in the depth direction.
 5. The image processing apparatus according to claim 1, wherein the setting unit is configured to newly set a search range for the portion whose position has not been identified based on a depth position of layer boundary whose position has been identified by the first identifying unit or the second identifying unit, and the second identifying unit is configured to further identify the position of a layer boundary portion whose position has not been identified based on a luminance change value in the depth direction in the search range having been newly set.
 6. The image processing apparatus according to claim 1, wherein the setting unit is configured to set a search range for an unidentified portion of a layer boundary that is identical to the layer boundary whose position has been identified by the identifying unit.
 7. The image processing apparatus according to claim 1, wherein the second identifying unit is configured to identify the position of the layer boundary portion based on a position where a luminance change in the depth direction is greater than a predetermined threshold value or a position where the luminance in the depth direction becomes a local maximum.
 8. An optical coherence tomography imaging system comprising: the image processing apparatus defined in claim 1; an optical coherence tomography imaging unit configured to obtain a tomography image of a target to be captured; and a display unit configured to display the tomography image together with the position of each layer boundary having been identified by the first identifying unit and the second identifying unit.
 9. A computer-readable storage medium storing a program that causes a computer to control an image processing apparatus, the program comprising: computer-executable instructions for identifying the position of at least a part of a layer boundary based on a tomography image of a target eye to be captured; computer-executable instructions for setting a search range for a portion whose position has not been identified, based on a depth directional position of the layer boundary whose position has been identified; and computer-executable instructions for identifying the position of a layer boundary portion whose position has not been identified by the first identifying unit based on a luminance value in a position where the layer boundary identified by the first identifying unit does not exist in the search range having been set, wherein the layer boundary is any one of an interface of an inner limiting membrane, an interface of a nerve fiber layer, an interface of an inner plexiform layer, an interface of an outer plexiform layer, an interface between inner and outer segments of the photoreceptors, or an interface of a retinal pigment epithelium.
 10. An image processing method comprising: causing a first identifying unit to identify the position of at least a part of a layer boundary based on a tomography image of a target eye to be captured; causing a setting unit to set a search range for a portion whose position has not been identified by the first identifying unit, based on a depth directional position of the layer boundary whose position has been identified by the first identifying unit; and causing a second identifying unit configured to identify the position of a layer boundary portion whose position has not been identified by the first identifying unit based on a luminance value in a position where the layer boundary identified by the first identifying unit does not exist in the search range having been set, wherein the layer boundary is any one of an interface of an inner limiting membrane, an interface of a nerve fiber layer, an interface of an inner plexiform layer, an interface of an outer plexiform layer, an interface between inner and outer segments of the photoreceptors, or an interface of a retinal pigment epithelium. 